Training recurrent network with block-diagonal approximated Levenberg-Marquardt algorithm
نویسندگان
چکیده
In this paper, we propose the block-diagonal matrix to approximate the Hessian matrix in the Levenberg Mar-quardt method in the training of neural networks. Two weight updating strategies, namely asynchronous and synchronous updating methods were investigated. Asyn-chronous method updates weights of one block at a time while synchronous method updates all weights at the same time. Variations of these two methods, which involves the determination of the parameters and , are examined.
منابع مشابه
A Layer-by-Layer Levenberg-Marquardt algorithm for Feedforward Multilayer Perceptron
The error backpropagation (EBP) algorithm for training feedforward multilayer perceptron (FMLP) has been used in many applications because it is simple and easy to implement. However, its gradient descent method prevents EBP algorithm from converging fast. To overcome the slow convergence of EBP algorithm, the second order methods have adapted. Levenberg-Marquardt (LM) algorithm is estimated to...
متن کاملCSLMEN: A New Optimized Method for Training Levenberg Marquardt Elman Network Based Cuckoo Search Algorithm
RNNs have local feedback loops within the network which allows them to shop earlier accessible patterns. This network can be educated with gradient descent back propagation and optimization technique such as second-order methods; conjugate gradient, quasi-Newton, Levenberg-Marquardt have also been used for networks training [14, 15]. But still this algorithm is not definite to find the global m...
متن کاملAn accelerated Levenberg-Marquardt algorithm for feedforward network
This paper proposes a new Levenberg-Marquardt algorithm that is accelerated by adjusting a Jacobian matrix and a quasi-Hessian matrix. The proposed method partitions the Jacobian matrix into block matrices and employs the inverse of a partitioned matrix to find the inverse of the quasi-Hessian matrix. Our method can avoid expensive operations and save memory in calculating the inverse of the qu...
متن کاملBearing Fault Diagnosis under Variable Speed Using Convolutional Neural Networks and the Stochastic Diagonal Levenberg-Marquardt Algorithm
This paper presents a novel method for diagnosing incipient bearing defects under variable operating speeds using convolutional neural networks (CNNs) trained via the stochastic diagonal Levenberg-Marquardt (S-DLM) algorithm. The CNNs utilize the spectral energy maps (SEMs) of the acoustic emission (AE) signals as inputs and automatically learn the optimal features, which yield the best discrim...
متن کاملModified Levenberg-Marquardt Method for Neural Networks Training
In this paper a modification on Levenberg-Marquardt algorithm for MLP neural network learning is proposed. The proposed algorithm has good convergence. This method reduces the amount of oscillation in learning procedure. An example is given to show usefulness of this method. Finally a simulation verifies the results of proposed method. Keywords—Levenberg-Marquardt, modification, neural network,...
متن کامل